Contrasting probabilistic scoring rules
نویسندگان
چکیده
منابع مشابه
Variogram-based proper scoring rules for probabilistic forecasts
Proper scoring rules provide a theoretically principled framework for the quantitative assessment of the predictive performance of probabilistic forecasts. While a wide selection of such scoring rules for univariate quantities exists, there are only few scoring rules for multivariate quantities, and many of them require that forecasts are given in the form of a probability density function. The...
متن کاملContrasting Essay Scoring 1 Running head: CONTRASTING ESSAY SCORING Contrasting State-of-the-Art Automated Scoring of Essays: Analysis
This study compared the results from nine automated essay scoring engines on eight essay scoring prompts drawn from six states that annually administer high-stakes writing assessments. Student essays from each state were randomly divided into three sets: a training set (used for modeling the essay prompt responses and consisting of text and ratings from two human raters along with a final or re...
متن کاملCommittee Scoring Rules
We present and advertise the class of committee scoring rules, recently introduced as multiwinner analogues of single-winner scoring rules. We present a hierarchy of committee scoring rules (which includes all previously studied subclasses of committee scoring rules, as well as two new subclasses) and discuss their axiomatic properties (while focusing on fixed-majority consistency and monotonic...
متن کاملLocal Proper Scoring Rules
Scoring rules assess the quality of probabilistic forecasts, by assigning a numerical score based on the predictive distribution and on the event or value that materializes. A scoring rule is proper if it encourages truthful reporting. It is local of order λ if the score depends on the predictive density only through its value and its derivatives of order up to λ at the observation. Previously,...
متن کاملProper Proxy Scoring Rules
Proper scoring rules can be used to incentivize a forecaster to truthfully report her private beliefs about the probabilities of future events and to evaluate the relative accuracy of forecasters. While standard scoring rules can score forecasts only once the associated events have been resolved, many applications would benefit from instant access to proper scores. In forecast aggregation, for ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Statistical Planning and Inference
سال: 2013
ISSN: 0378-3758
DOI: 10.1016/j.jspi.2013.05.012